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ABSTRACT 


We evaluate the effectiveness of small high school reform in the country’s largest school district, 
New York City. Using a rich administrative dataset for multiple cohorts of students and distance 
between student residence and school to instrument for endogenous school selection, we find 
substantial heterogeneity in school effects: newly created small schools have positive effects on 
graduation and some other education outcomes while older small schools do not. Importantly, 
we show that ignoring this source of treatment effect heterogeneity by assuming a common small 
school effect yields a misleading zero effect of small school attendance. 

Key words: education reform, small schools, heterogeneous treatment, instrumental variables 
JEL Codes: 121,128 


INTRODUCTION 


While the academic achievement of U.S. elementary school students has improved 
over the last decade, U.S. high school students continue to graduate at unacceptably low rates 
and measures of achievement show only a slight upward trend since 2005 (National Center for 
Education Statistics, 2010). Moreover, the achievement and graduation gaps between white and 
black high school students and between white and Hispanic high school students have not 
changed. For example, by some calculations slightly over 80 percent of white students graduate 
within four years, but only 60 percent of black and 62 percent of Hispanic students do so 
(Stillwell, 2009), and the gap in college readiness is similarly stark (Greene and Foster, 2003). In 
addition, with the erosion of the labor market for low-skilled workers over the past several 
decades, the gap in earnings between high school graduates and non-graduates has increased 
(Day and Newburger, 2002). Within this context many school systems with large proportions of 
poor students, in particular large urban school systems, face tremendous challenges; a majority 
of their students are at risk of not succeeding in high school and thus have more limited access to 
post-secondary education and have lower labor market earnings than many of their counterparts 
in suburban districts. While several reforms target high school students, the small school reform 
stands out because of its adoption in many major cities and its substantial public and 
philanthropic funding base. Placing students in small schools is advocated as a way to provide 
students with the support they need to improve their performance. 

There have been at least two waves of small high school refonn in U.S. cities as well 
as an early and more recent literature on their effectiveness. The early wave of small school 
reforms in cities such as New York City (NYC), Chicago, Philadelphia and Oakland occurred in 
the early 1990s. The early literature that stimulated and accompanied these reforms was 
conceptual (establishing theoretical reasons why small schools would help disadvantaged youth) 
and, when empirical, correlational in nature. A later wave of small school refonn occurred after 
2000 in NYC, Chicago, Fos Angeles, San Diego, Philadelphia, and Boston among others, often 
with some funding from large foundations such as the Gates Foundation, the Carnegie 
Corporation, and the Open Society Institute as well as the U.S. Department of Education (U.S. 
Department of Education, 2006). The literature on this wave includes studies based on causal 
modeling using econometrics or, in one case, a lottery design (Bloom et al 2010). 
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In this study, we use administrative data covering all NYC public high school students 
to evaluate the effectiveness of two generations of small schools in NYC. The long recognized 
challenge in educational evaluations is the possible selection of students into the education 
intervention, which can bias simple comparisons of outcomes for those who are treated by the 
intervention and those who are not. In our application, selection bias arises if students who 
attend small and large schools differ on dimensions, such as motivation, ability, and parental 
support, which have an independent effect on the outcomes of interest. We address selection 
bias in two ways. First, we use a rich set of student characteristics, such as gender, race, 
language skills, prior middle school test scores etc., to control for many of the observable 
differences between students attending small and large high schools. But, as in a wide variety of 
evaluation contexts, the observed student characteristics in our data are unlikely to fully 
eliminate unobserved or unmeasured differences in student characteristics that affect student 
outcomes. 

Recognizing this potentially important selection on unobservables, we next turn to 
quasi-experimental methods using credible instrumental variables that exogenously influence 
student decisions to attend small schools but do not influence student outcomes. Since high 
schools of various sizes are not evenly distributed across the city, and students who live in the 
immediate vicinity of a small high school (especially relative to a large school) are more likely to 
attend a small school, we use as instruments the distance between the nearest small school or 
large school and the student’s home. 

Motivating our use of distance as an instrument variable is a small but growing 
literature on the determinants of school choice. 1 A consistent result in the literature is that 
location (and specifically distance) of a school relative to a student’s home residence is an 
important variable for students and parents in their choice of school. Schneider and Buckley 
(2002) report that in parent internet search behavior, location is the second most sought after 


Studies that rely on survey analysis indicate that parents generally choose based on academic achievement and the 
quality of teachers at the school (Armor and Peiser, 1998; Greene et al., 1998; Kleitz et ah, 2000) and do not exhibit 
much preference towards student demographics (Schneider, Marschall, Teske, and Roch, 1998; Howell, 2004). In 
contrast to survey studies, observed choice behavior reveals that parents do have strong preferences for schools with 
similar demographics (Chakrabarti and Roy, forthcoming; Glazerman, 1997; Schneider and Buckley, 2002; 

Saporito, 2003; Hastings, Kane and Staiger, 2006). There is also literature on the impacts of choice, for example on 
socioeconomic and racial sorting and segregation (Hoxby, 2003; Ladd, 2003) and on decisions between private and 
public school (Epple and Romano, 2002; Hoyt and Lee, 1998). 
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piece of information after school demographics. Burgess and Briggs (2010), in a study of 
parental preferences for schools in England, conclude that parents make tradeoffs among 
academic attainment, school socio-economic composition, and travel distance. Hastings, Kane 
and Staiger (2006) find that in North Carolina proximity is highly valued by all, although 
families with strong preferences for academics are generally willing to tolerate longer distances. 
Saporito and Lareau (1999) conclude that both whites and blacks tend to choose schools close to 
their homes but whites are often willing to travel further to attend schools with higher 
proportions of white students. Motivated by this prior literature, we fonn instruments from the 
distance between the nearest small or large school and the student’s home. A similar 
instrumental variables framework has been used in an educational evaluation of Chicago schools 
(Cullen et ah, 2005), an evaluation of small schools (Barrow et ah, 2010) and charter schools 
(Booker et ah, 2011) in Chicago, and an examination of the effect of college attendance on 
earnings (Card, 1995) and on health behaviors (Currie and Morretti 2003). As this prior research 
has demonstrated in a variety of contexts, the likelihood of attending a school decreases as the 
distance to the school increases, perhaps because of higher costs such as those involving 
transportation. 

We confirm these results with our NYC data and show that distance strongly predicts 
actual small school attendance, even after conditioning on student characteristics. We also 
present several additional analyses that support the instrument exogeneity. We use these distance 
based I Vs to instrument for small school attendance and obtain IV estimates of causal effects of 
attending small schools. Suggesting the importance of student sorting into schools based on 
unobserved student characteristics such as motivation, we find a positive effect of small school 
attendance with the OLS estimator but a small and imprecise estimate using the IV estimator. 

An important contribution of this paper is to distinguish between the old and new 
generation of small schools. Rather than assume a common small school effect, we instead 
divide the small schools into those newly developed since 2002 and those which existed prior to 
the latest wave of reforms. These new small schools are different in a number of ways from the 
old small schools, differences that we further explore to assess whether they are related to 
effectiveness and whether they can or will be sustained. 

Using models where we distinguish between new and old small schools, we find 
important differences in the effects of the schools in both our OLS and IV estimates. In our IV 
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estimates, using instruments for distance to the new and old small schools, attending an old small 
school is estimated to have a negative effect on the probability of graduating relative to large 
schools, while attending a new small school is estimated to increase graduation rates by 17 
percent relative to attending a large school. This estimate is statistically significant from zero at 
the five percent level and the magnitude of the estimate is robust to changes in sample selection, 
variable definitions, and various alternative instrumental variable estimators. When we turn to 
other high school outcomes, we find more mixed results. Our IV estimates indicate that 
attending a new small school increases the probability of taking the Regents English and 
mathematics examinations by 14 and 16 percent respectively. We estimate, however, a negative 
effect of new small school attendance on English scores and no effect on mathematics scores. 

We cannot rule out, however, that the non-positive effects on test scores is the result of the 
marginal test takers induced to take these exams having lower ability. In addition, we estimate 
that old small schools have considerably more negative effects on test scores than the new small 
schools. 

Our estimates reveal a clear divide in the effects of the new versus old small schools 
and provide some context to understand the results of previous research. Studying a subset of 
the new small schools, which were over-subscribed and offered admission by lottery, Bloom et al 
(2010) find a 6.8 percentage point increase in graduation rates from attending these “small 
schools of choice.” Our positive statistically significant effect of new small school attendance 
estimated using a different empirical strategy - the IV — is consistent with this finding, and given 
the standard errors, both estimates are within the other’s 95 percent confidence interval. An 
advantage of our study is that we can use our identification strategy to estimate the effect of a 
wider variety of small school types. Thus, our estimates show that the positive effects estimated 
for the recent small schools would not necessarily extrapolate to all small schools. This is a 
crucial finding for policy: school size matters but it is not sufficient for affecting outcomes. 

The paper is organized as follows. In section II, we review the literature on small high 
school reform and situate our contribution in this literature. In section III, we describe the NYC 
context for our evaluation. In section IV, we describe our data and measures, in section V, we 
present our models and methods, and in section VI we present results. In section VII, we discuss 
why we might find our results and in section VIII, we conclude with a discussion of the 
relevance of the results for policymakers. 
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PREVIOUS LITERATURE AND OUR CONTRIBUTION 


What is Small? 

It is important to note is that there is no consensus on a definition of “small” in the 
literature on school size and outcomes. The federal government, through its Small Schools 
Initiative, set a limit of 300 students (U.S. Department of Education, 2006) while the Gates- 
funded initiative in NYC considered 500 students the upper limit for small high schools 
(Gootman, 2005) and a recent study in Chicago established a 600 student cutoff (Barrow et ah, 
2010). In previous research in the mid-1990s on the costs of small high schools in NYC, as well 
as the then-current local policy, schools with 600 students or fewer were considered small 
(Stiefel et ah, 2000). Lee and Smith (1997) found schools in the range of 600 to 900 to be most 
effective for minority students, and finally, a recent Gates funded study (Bloom et ah, 2010) used 
550 as the cutoff for a small school. To incorporate a policy-relevant figure in the range of the 
literature, we focus first on a 550-student cutoff and then perform a sensitivity analysis to 
ascertain the effect of alternative small sizes. 

Early Literature on Small High Schools 

Despite the lack of consensus on what constitutes a small school enrollment, the early 
literature proposed many hypotheses about how small size could affect student outcomes. 

Fowler (1992) and Page et al. (2002) advanced the idea that small schools have more student 
participation in extracurricular activities and better student and teacher attitudes. Others 
hypothesized that small schools are particularly effective for disadvantaged students as a result 
of their superior social aspects, high perceived expectations for all students, teacher and 
administrator abilities to nurture students’ (higher) needs, and better student behavior (Barker 
and Gump, 1964; Lindsay, 1982; Gottfredson and Gottfredson, 1985; Entwisle, 1990; Haller, 
1992; Lee and Burkham, 2003). 

But the literature is not unified on the directions of effects, as some authors have 
claimed that increased numbers of academic offerings and a social climate that is more accepting 
of diversity are more likely in large schools (Pittman and Haughwout, 1987; Haller et al., 1990; 
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Watt, 2003). Moreover, high school reform in the mid 20 th century featured the substitution of 
large high schools for small ones (Conant, 1959). 

Empirical work before 2000, which was largely based on correlations and ignored 
issues of selection, suggested that achievement scores and attendance rates were higher and 
dropout rates were lower in small schools compared to large schools (Fowler and Walberg, 1991; 
Fowler, 1992; Lee and Smith, 1997). One of the most rigorous studies of student outcomes 
found that an optimal school size with respect to maximizing student achievement ranged 
between 600 and 900 students (Lee and Smith, 1997). 

Another focus of the early research was heterogeneity in the effects of small schools 
for particular sub-groups defined by observed student characteristics. According to Fowler and 
Walberg (1991) and Fowler’s (1992) literature reviews, schools with fewer than about 1,500 
deliver superior outcomes for minority and poor youth. Large schools may also have particularly 
negative effects on disadvantaged students, and small schools may better serve disadvantaged 
students not only vis-a-vis absolute achievement levels (Flowley et ah, 2000; Bickel et ah, 2001), 
but also with respect to lessening of achievement gaps (McMillen, 2004). Some empirical 
analyses, though, suggested that small schools provide benefits to all student types and that the 
distribution of gains across socioeconomic status and race is more equitable in smaller schools 
(Lee and Smith, 1995). None of these studies, however, addressed the potential selection bias or 
endogeneity of the choice to attend a small school based on unobservable student ability, 
motivation, parental involvement etc. 

More Recent Literature 

More recently, scholars have turned their attention to the causality issue and have 
expanded their use of statistical methods and experimental designs to address it. Schneider et al. 
(2007) evaluated small school effects using data from the Educational Longitudinal Study of 
2002 (ELS: 2002). They compared estimates from a random coefficients/hierarchical 
longitudinal model (HLM) to those from a propensity score matching estimator using the 
available observable covariates. The authors found with both methods that attending a small 
high school has little effect on achievement, with the HLM estimates showing somewhat larger 
effects than the matching estimates for post-secondary expectations and number of colleges to 
which students applied. Both the HLM and matching frameworks have shortcomings in 
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addressing selection, however. HLM, a type of control function approach, assumes a particular 
specification of the nested structure for outcomes. This approach, like parametric control 
function approaches, is not robust to model mis-specification. The matching framework, while 
potentially more flexible than some standard OLS regression frameworks, assumes that selection 

into small schools occurs only through the observable covariates available in the particular 

2 

dataset used. Selection based on unobservables could bias these results. 

Barrow et al. (2010) used quasi-experimental variation in the distance between 
students’ homes to high schools in an instrumental variables (IV) framework to evaluate the 
effect on performance of attending small high schools in Chicago. In their IV results, they found 
a positive effect of small school attendance on continuation through high school and graduation, 
but their study included only 22 small high schools and could not distinguish among school 
vintages. 

Focusing on what Bloom et al. (2010) labeled “small schools of choice (SSC),” which 
enroll fewer than 550 students, the authors took advantage of lotteries for some seats in 
oversubscribed schools instituted by NYC’s high school application processing system starting 
with the class entering ninth grade in 2005. Over the years 2005 through 2008, 105 such 
schools, serving students in grades 9-12 and founded after 2002, were part of the study’s sample. 
Students not lotteried into a particular school might have been lotteried into another SSC. 
Therefore, the students not lotteried into any SSC and who attended another NYC high school, 
not necessarily a large one, served as the control group. Students in one cohort (2005) were 
followed for four years and had statistically significantly higher graduation rates (6.8% higher) if 
they enrolled in SSC’s than if they did not. Other cohorts, not in the schools for four years at the 
time of the study, had more credits toward graduation each year in SSCs than in other schools. 
The study provides some limited evidence on the effectiveness of a select group of small high 
schools - ones that were newly formed after 2002 and that were oversubscribed for some of their 
seats - that is, popular, new small schools. It does not provide evidence, however, on the 
broader issue of whether size itself is the important feature of these schools (since their sample 
does not include older small schools or undersubscribed small schools) and it does not provide a 


2 

~ See Heckman and Navarro (2005) for a discussion of the difference in identifying assumptions for matching versus 
control function approaches. 
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clear counterfactual since students may have attended other small schools if not lotteried into 
their choices. 

In summary, our contributions involve the use of rich student- and school-level 
administrative data to study the effects of small high school attendance across multiple cohorts of 
students, attending several generations and types of small schools, the use of IV methods to 
obtain causal estimates, sensitivity to the definition of size, and the evaluation of multiple 
outcomes. 

CONTEXT OF SMALL SCHOOL REFORM IN NYC 


New York City is a particularly useful and relevant setting to study the effectiveness 
of small high schools. It is a large, ethnically diverse urban school system, which make the 
results relevant to other urban settings that face the challenge of educating at-risk students. NYC 
also provides a large sample of small high schools, which enables us to examine the relationship 
between size and performance within the sample of small schools, in addition to comparing small 
and large high schools. 

In 2002, the New York State legislature granted Michael Bloomberg, the newly 
elected mayor of NYC, control of the NYC Public Schools. Mayor Bloomberg hired Joel Klein 
to be Chancellor, charging him with improving significantly the perfonnance of NYC’s one 
million plus public school students. One of Chancellor Klein’s major initiatives was to establish 
new small high schools, replacing large dysfunctional ones. Over time, this strategy also 
involved providing high school students with a portfolio of schools from which they could 
choose to attend via an elaborate selection process modeled on the physician residency 
placement model. 


3 The High School Application Processing System (HSAPS), was introduced in NYC in the 2003-2004 academic 
year. HSAPS requires all NYC public school 8 th grade students to submit to the NYC Department of Education 
(NYCDOE) a list of up to 12 schools ranked in order of preference. The NYCDOE then uses a computerized 
process, modeled on the physician residency placement model, to assign students to their highest-ranked school 
whose admissions criteria they meet and where spaces are available. The matching takes into account students’ 
preferences and schools’ selection criteria and is supposed to be immune to gaming, thus encouraging students to list 
schools in order of their true choices (Abdulkadiroglu, Pathak and Alvin E. Roth, 2005, 2009). Several high schools 
require entrance examinations, which students often take in the fall of their 8 th grade year; students who qualify 
based on their test scores are offered places before the due date for rankings of other schools. In our empirical work, 
we perform robustness tests that exclude such schools to make sure that they are not driving results. 
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Chancellor Klein and his team succeeded in opening a very large number of new high 
schools. 4 Table 1 shows the numbers and characteristics of NYC high schools in 2007 an 2008 
based on our sample of students and schools, which is described more fully below and includes 
the majority of non-special education high schools in these years. This sample is made up of two 
cohorts of NYC high school students who were scheduled for on-time (four-year) graduation in 

2007 or 2008, were in the NYC public schools in 8 th grade, attended non-special education 
schools, and attended schools with four grades by their graduation year. Note the large number 
of small schools (defined here as having 550 students) and student enrollment in small schools of 
just over 20% of our sample. In terms of the creation of new small schools, by 2008 there were 
121 of them. 5 Figure 1 shows the distribution of schools in our sample by size for the 2007 and 

2008 cohorts compared to similarly constructed cohorts for 2001 and 2001 and clearly illustrates 
how much the size distribution was changed to favor small schools over these Klein-Bloomberg 
years. 

Note that the NYCDOE set up a different creation process and regulatory environment 
for the new small schools established after 2002 compared to the old small schools that had been 
in existence before that time (Cahill and Hughes, 2010; Bloom et ah, 2010). First, the new small 
schools came into being through a competitive application process in which school organizers 
proposed how they would institute academically rigorous curricula and partner with community 
organizations and not all applications were successful. Second, they were almost all supported by 
non-profit organizations, often New Visions for Public Schools (http://www.newvisions.org/), 
which were funded by the Gates and some other Foundations to monitor, aid and network them 
with other new small schools as they became established. Third, they were given some 
exemptions in their first years from serving some groups of special needs students and following 
all union rules on hiring teachers, although some of these exemptions were to disappear once 
they were fully functional. Last, their principals were often trained by an organization that was 
bom through a NY CDOE effort to train new leaders who embraced accountability and 

4 According to Bloom et al (2010), between 2002 and the time of their report (2010), NYC “. . . closed more than 20 
underperforming public high schools, opened more than 200 new secondary schools. . ..(p.12)” These numbers 
differ from ours in part because they run through 2010 while ours go through 2008 only. 

5 In this paper, a new school is one that had no graduates in the 200 1 or 2002 cohorts but did have graduates in the 
2007 or 2008 cohorts. A new school could have begun as early as 1999-2000, but generally new schools began with 
only a 9 th grade class, adding one grade per year, thus not becoming full high schools until 2002-2003. The majority 
of schools labeled new began in 2002 or later (over 60% overall and over 62% for new small schools). 
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empowerment by schools. 6 Later in this paper, we explore whether these differences were 
reflected in differences in quantitative characteristics of new small schools compared to existing 
small schools, such as their student composition, their teacher/student ratios, their per pupil 
expenditures, or their teacher characteristics. 

A key concern of policy makers is the four year graduation rate. As Figure 2 
demonstrates there is a tremendous variation in the school-level graduation rate across the high 
schools in NYC. While some high schools achieve above 90 percent graduation rates, other 
schools see less than 40 percent of their students graduate. Thus, there is substantial 
heterogeneity in graduation rates across schools. 

DATA AND MEASURES 


We use student- and school-level data from the NYC Department of Education 
(NYCDOE) administrative datasets. The student-level data are drawn from a census of NYC 
public high school students expected to graduate in either 2007 or 2008. Our sample of high 
schools includes those attended by the cohorts of students, with the exception of specialized 
program schools (such as last chance high schools or schools for pregnant mothers) or schools 
with predominantly full-time special education students. We exclude schools and students 
located on Staten Island because no small high schools exist there and students generally did not 
travel outside Staten Island to attend a small high school. 

The student data include student characteristics, such as socio-economic status, 
demographics and educational program participation, as well as a number of outcomes. Whether 
a student participated in the free lunch program in 8 th grade serves as an indicator of poverty 
status, 7 and other demographic and education program variables include race/ethnicity, gender, 
whether or not a student was an English language learner, whether a student is overage for grade, 
and prior test scores in 8 th grade reading and mathematics, which we convert to z scores with 
mean zero and standard deviation one for each cohort in our sample. Also part of the student and 
school data are residence zip codes and school addresses, which enable us to calculate Euclidean 
distance, in miles, between home and schools. 

6 See Corcoran et al (2012) for more on the leadership academy. 

7 We use an indicator of whether a student was eligible for free lunch in 8 lh grade rather than if she/he is eligible in 
high school, because the latter, unlike the former, is a notoriously poorly-reported variable. 
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Student high school outcomes include both graduation outcomes and high school test 
scores. Graduation is defined as earning a local, Regents, Honors or Advanced Regents diploma 
in four years. We exclude GEDs from the definition of graduation, but conduct robustness tests 
classifying obtaining a GED as graduation. The school we assign to students is the school in 
which they are enrolled in 9 th grade. Although students are able to transfer among high schools, 
we assign them to their 9 grade school in order to obtain an “intent to treat” estimate. 

Students in NYC’s public high schools must take statewide Regents’ examinations in 
a number of subject areas in order to receive a diploma. We focus on the Regents’ English and 
mathematics exams, as these were the first required of all students as part of the state’s new 
graduation requirements that began in 1999-00. We measure whether students take these exams, 
and conditional on taking them, whether their scores meet the cutoffs for various kinds of 
diplomas, cutoffs that changed over the time of our study. For the 2007 and 2008 cohorts, a 
minimal “local” diploma required a minimum score of 55 in any one of five core Regents’ 
areas, 9 and the higher level Regents’ diploma required a minimum score of 65 on all five of these 
exams. In addition, an Advanced Regents’ diploma was available to these later cohorts with 
scores of at least 65 on eight Regents’ exams. (See Appendix A for a more complete explanation 
of the Regents graduation requirements.) 

School size is defined by the total number of students enrolled at each high school in a 
student’s 9 th grade year based on data from NYCDOE School Based Expenditure Reports and 
New York State Annual School Reports. Our primary definition of a small school is one with 550 
or fewer students enrolled, although we analyze effects for larger sizes as well. Our reason for 
choosing this size is to be in the range of other studies and, in particular, to be consistent with a 
recent study in NYC, reviewed earlier (Bloom et ah, 2010). A new school is one that had no 
graduates in the 2001 or 2002 cohorts but did have graduates in the 2007 or 2008 cohorts. 

Table 1 displays the descriptive statistics for the combined 76,213 students in our 
study. 10 These students all began high school in 9 th grade and are included in the graduation (or 
non-graduation) statistics. As discussed above, the number of small schools and enrollment in 
them increased substantially since 2001 and by the time of our study there were 169 of them. 
Comparing the descriptive statistics for small versus large high schools, we see that small 

s About 15-20 percent of the 2007-08 cohort of students changed high schools. 

9 The five core Regents’ exam areas are English, science, math, U.S. history and government, and global studies. 

10 Appendix Table B.l contains definitions of each variable. 
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schools overall have a more disadvantaged student population on a variety of measures. Small 
high schools have a higher proportion of their students eligible for free or reduced price lunches 
than do large high schools and more students overage for grade. Small high schools students 
also score lower on 8 th grade tests than students in large high schools. 11 The one exception to 
this pattern is that small school students have a higher proportion with English as a home 
language. Previewing our regression estimates described below, small schools have a higher 
overall graduation rate and a higher rate of students taking core mathematics and English 
Regents exams, but a lower or similar rate of achieving high scores (> 55 or > 65). 

In summary, in both of the cohorts the composition of students in small and large high 
schools differs, with small high school students being generally less advantaged, making it 
essential to control for these characteristics in models of the effects of small school attendance 
on high school outcomes. For example, low-performing students may be more likely than other 
students to attend small schools, perhaps with the hope that less mainstream schools will turn 
their performance around, or for some other unobserved reason related to performance. These 
students may experience gains in the small schools, yet continue to perform at lower levels than 
their large school counterparts. If that is the case, the average performance of students in these 
small schools compared to the rest of the schools will be lower, although the causal effect of 
small schools could still be positive. 

MODELS AND METHODS 

Student Performance Model: Common Small School Effect 

Following the previous literature, we specify a stylized educational production function in 
which we model student outcomes as a function of observable variables capturing student socio- 
demographic and educational characteristics, performance on eighth grade English and math 
tests, and borough of residence, since in NYC these boroughs differ in their population income, 
education, and other demographics and could influence small school attendance and 
performance. Our basic model expresses student performance as follows: 

(1 )Perf ijt - a 0 + a x Small i/t + STf a 2 + Test8 f a 3 + Boroughf a 4 + s tp 


ii 


By construction, the overall mean test score in our sample is zero (and the standard deviation is one). 
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where Per% is a student outcome (such as earning a diploma within four years or taking a 
Regents’ examination) for student i in school j in year t. Since we use cohort data, as opposed to 
panel data, there is only one observation per student in each cohort dataset. Small jj t is an indicator 
that takes a value of 1 if, in year t, student i attended a school j that is small, as measured in this 
basic model, by a school that enrolls 550 or fewer students. ST it is a vector of student 
characteristics, including gender, race/ethnicity, free lunch status, English language proficiency, 
and overage for grade. Test8i t is a vector of eighth grade reading and mathematics exam scores 
(each score, each score squared, and interacted), and Borough^ is a set of indicators for the 
student’s borough of residence (borough fixed effects). The a ' s are a set of intercepts and 
slopes that capture the impact of the corresponding variables on student performance, with a x , in 
particular, capturing the average difference in performance between students who attend small 
schools and students who do not, controlling for student characteristics and past performance. Sij t 
represents the remaining variation. All standard errors are appropriately modified to reflect 
possible heteroskedasticity and clustering of students at the school level. 

Heterogeneity in Treatment: New Versus Old Small Schools 

Model (1) imposes two forms of homogeneity assumptions: i) it assumes that the 
effect of small schools is the same for all students, and ii) it assumes that the effect of small 
schools is the same for all types of small schools. Like much of the existing literature, we 
explore the student level heterogeneity (i) by estimating the effect of small schools for sub- 
groups defined by observable student characteristics. We focus here, however, on the school 
level heterogeneity by expanding (1) to allow for different effects of small school by the 
“vintage” of the school: 

(2 )Perf jjt = a 0 + /?, OldSmall lJt + j3 2 New Small ijt +ST it 'a 2 + TestS it 'a 3 + Borough it ' a 4 + s i]t where 
/?! is the effect of attending an old small school (defined as schools in operation prior to 2001-02) 
and fd 2 is the effect of attending a new small school (defined as schools which began operations 
in 2001-02 or after). Since the new small schools differed in substantial ways from the old small 
schools (as discussed above), distinguishing between the effects of the different types of schools 
allows us to isolate whether school size is the key school characteristic for student performance 
or whether other features of the school are important. 


IESP Working Paper #1-12 13 


Instrumental Variable Strategy 

In order to overcome the possible selection of students into small schools on the basis 
of unobservable characteristics, we instrument for small school attendance with variables that 
plausibly affect school attendance but do not directly affect student outcomes except through 
small school attendance. As described earlier, these instruments are based on the minimum 
Euclidian distance between the nearest small school (either new or old small schools) or large 
school and the student’s 8 th grade residence zip code. 12 Note that we include borough fixed 
effects in the main specification so that we control for unobserved factors correlated with 
distance at the borough level. 

First Stage of IV Estimations 

Table 2 reports linear probability models providing the relationship between small 
school attendance and various measures of distance. The exact first stages for our main results 
(discussed below) are reported in the Appendix Table B.2. We also discuss below a series of 
robustness exercises to test whether our main results are sensitive to various configurations of the 
distance instruments. 

In Table 2, the coefficients on the variable measuring distance from a small school and 
its square are statistically significant and plausible. Comparing Columns (1) and (2) we see that 
there is a concave relationship between distance and attending a small school with a negative 
linear term and positive quadratic term. For example, the probability of attending a small school 

IT 

as distance increases by one mile (from 0 miles away) decreases by 10%, conditioning on 
student covariates and dummy variables for NYC boroughs. In contrast, distance to large school 
does not have a statistically significant effect of small school attendance in Column 2. As 
discussed below, in a series of robustness exercises, we also use distance to large schools as a 
control variable in the main specification, thus giving the distance to small school variables a 
relative distance interpretation. Our main results are robust to this instrument and control 

12 We use 8 th grade home zip code for 2007 cohort. For 2008, only 7 th grade home zip codes were available; home 
addresses were not available for any cohort. We calculate the distance between the centroid of each zipcode and the 
school address. Distances are calculated using Stata 1 1 vincenty code. There are 170 student residence zip codes in 
our sample. 

13 Predicted change in probability of small school attendance from 0 to 1 mile away from school is 
-0. 115*1+0. 015*1 A 2 = -0.1. 
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variable configuration. In Appendix Table B.2, we show that distance to various vintages of 
small schools, new or old small schools, also has a statistically significant relationship with 
attendance at these school types. Across our various IV specifications, the F-statistic for the total 
regression or for the excluded instruments is large, indicating that our distance instruments 
provide strong instruments for small school attendance (Staiger and Stock, 1997). 

Threats to Validity 

There are potentially a number of different threats to the validity of the distance based 
IV strategy. We discuss each of these in turn and when possible provide some evidence on their 
importance. 

Location of Schools'. The first issue is that the location of the small schools could be 
correlated with unobservable characteristics, but there are institutional reasons to think that this 
might not be the case. Many small schools were co-located with other small schools in buildings 
vacated by very large schools. These large, vacated schools had been existence for awhile and 
were largely exogenously set with respect to unobservable student characteristics. For example, 
while some large schools are co-located, small schools are significantly more likely to be sharing 
space with almost sixty percent of small schools sharing space for the 2008 cohort. Co-location 
is particularly prevalent among the new small schools: over two-thirds of new small schools 
share space with another school. 

In addition to institutional reasons to suspect that the location of small schools is 
largely unrelated to student characteristics, we can indirectly examine the exclusion restriction 
by studying the correlation of school location with observable student characteristics. Appendix 
Table B.3 regresses the minimum distance of students by zip code to small schools, new small 
schools, and old small schools as a function of the average level of various student level 
characteristics at the zip code level, such as average racial composition, proportion of students 
receiving free lunch, and average 8 th grade mathematics and reading test scores. 14 In each of 
these regressions, we find that minimum distance in a zip code to small schools is unrelated to 
the average characteristics of the zip code’s student residents for all small schools (Column 1) 
and for new small schools (Column 2). For old small schools (Column 3), we find some patterns 


14 Recall that student residence location is at the centroid of the zip code so each zip code has one minimum distance 
to each type of school. 
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- between proportions of black and white students as well as student 8 th grade math z score and 
minimum distance. There is, however, no relationship with 8 th grading reading scores or 
poverty. We also find that distance to alternative school types (e.g. large schools) is often a 
predictor of distance to small schools as schools of all types are clustered in areas with higher 
population density. 

While these results shed some light on the location of schools relative to the 
characteristics of the student population, these types of correlations are not a threat to the validity 
of our distance based IV strategy since all the student characteristics are observable in our 
sample and are included as control variables in our analysis. Whether school location is wholly 
unrelated to unobservable student characteristics is, to some extent, still an open question, one 
which we cannot resolve. Supporting the validity of distance type instruments, however, Cullen 
et al (2005) found in their study of Chicago public schools that distance to school was 
uncorrelated with a rich set of additional variables they collected, variables that would be 
unobservable in our administrative data. 

Student Mobility: A related second issue is that a student’s family might move to be 
closer to a particular school type, thereby creating a correlation between distance to school and 
unobservable student characteristics related to their perceived gains from theses schools. Our 
analysis above, which examined the relationship between average student characteristics across 
zip codes, suggested that whatever student mobility existed during our sample period, it was not 
sufficiently related to distance to school such that we estimate many statistically significant 
relationships between distance and the observable of the students residing in the zip code areas. 

In addition, although one might be concerned that families will move to locate near a 
desired high school, there is little in the high school application process or system of preferences 
that creates a strong incentive to do so and there is little empirical evidence of such mobility. An 
examination of the residential moves made by 8 th grade students in 2007 reveals that about nine 
percent changed zip code between 7 th and 8 th grade and those include both moves closer to and 
farther from small schools. We tested the relationship between these moves and school distances 
by regressing an indicator for student mobility on our set of student control variables and after- 
move distance to old and new small schools. (Results available from authors.) Controlling for 
student characteristics, we find no statistically significant difference in the distance to old or new 
small schools for individuals who move zip codes. This analysis suggests that for those students 
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who do move, we cannot reject the hypothesis at the 10 percent level that these moves were 
unrelated to the location of old and new small high schools. 

Student Entry and Exit : Another potential issue is that distance to various schooling 
options might have an impact on the set of students choosing to attend public high school, either 
causing some students to exit the NYC public system if they reside too far away from a desired 
high school option, or enter the NYC public system if they reside close to a desired option. Our 
student sample consists of only high school students who were also enrolled in NYC public 
middle schools. This sample restriction is necessary here in order for us to have available two of 
our key control variables, 8 th grade mathematics and reading test scores. Thus our sample does 
not include students who entered or left the NYC public system between middle and high school. 

It is important to emphasize that while distance to school types may be related to entry 
and exit decisions of students, this correlation does not affect the internal validity of the 
estimates we provide: within the sample of students who enroll in both NYC public middle and 
high school, distance to various school types is a plausible instrument. The issue of whether 
small school attendance would affect high school outcomes for the students who enter or exit the 
NYC system in the same way as we estimate for the population of students who remain in the 
system is fundamentally a question of external validity: Is the local average treatment effect 
(LATE) we identify representative for the whole population of high school students? The same 
extrapolation issue would of course apply to studies using lottery or discontinuities in admission 
criteria for identification as students may enter or leave the public system in response to the 
admission decisions (see Engberg et al 2010 for a discussion of these issues). 

School Competition Spillovers: Another possible threat to the validity of our IV 
strategy is that distance to particular school may influence the outcomes of students not attending 
that school through a school competition spillover effect. Nearby high performing schools may 
induce principals in other schools to change their behavior in some way to make their school 
more attractive to students. In NYC, this type of local, intra-district, competition is unlikely 
because the comparison groups by which school performance is judged are explicitly district- 
wide. NYC’s accountability system compares schools that are similar in student body 
composition but very often geographically distant. Thus, the extent of competitive pressures 
from geographic distance may be considerably less relevant than across district competition. 
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RESULTS 


Common Small School Model 

In our most basic model, we estimate the effect of attending a small high school on 
earning a diploma in four years, using a linear probability model. Estimation using a probit 
model yields similar results (available from authors), but the linear probability specification is 
reported for ease of interpretation. In Table 3, Column (1) we estimate our basic model using 
OLS, including control variables for a number of student demographic characteristics (gender 
and race indicators), English proficiency and free lunch poverty status, and a quadratic function 
of 8 th grade mathematics and reading scores (each score alone, squared, and their interaction). In 
addition, we include borough fixed effects. Controlling for these variables, the OLS estimated 
effect is positive and statistically significant from zero (p < .01). The coefficient estimate of 
0.108 indicates that students are nearly 1 1 percent more likely to graduate if they attend a small 
rather than a large high school. 

These results, however, are potentially suspect because, as discussed earlier, students 
may decide to attend small schools based on unobserved characteristics that could also 
predispose them to be more or less likely to earn a diploma, and such selection bias could affect 
the size and the sign of the coefficient on the small school indicator. To address this source of 
selection bias, we employ an instrumental variable approach, using a two stage least squares 
(2SLS), with instruments for small school attendance based on distance from the students’ homes 
to the nearest small and large high schools. The first stage estimates are reported in Appendix 
Table B.2 (see the discussion of instrument strength and validity above). 

Table 3 Column (2) displays the second stage IV estimates for the effect of small 
school attendance on earning a diploma in four years. As with the OLS results, all of these 
models include the full set of student covariates and borough fixed effects. The IV results differ 
dramatically from the OLS estimates as we now obtain a small and not statistically significant (at 
p < .10) effect of small school attendance. 

If we were to end the analysis here, we would draw the following conclusions. 
Although on observable characteristics more disadvantaged students attend small schools in 
NYC, there seems to be positive selection on unobservable variables into the small schools given 
the positive OLS estimates of small school attendance and the negative or insignificant from zero 
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IV estimates. After correcting estimates for selection on unobservables, using strong instruments 
related to distance from residence to school, the coefficient on the small school variable changes 
signs and significance, from positive in the OLS results to insignificant in the IV results. The 
unobservables that are affecting selection into small high schools are not knowable using our 
data, but they may plausibly be related to motivation and/or parental involvement. This result 
implies that it is not school size but rather selected students that make the OLS estimates of small 
school effects positive. Stopping here, however, might not present the entire story since small 
schools are not all created equal, outcomes other than earning a diploma can be important, and 
we have only explored one definition of size. We turn to these issues next. 

New versus Old Small Schools 

We next explore whether some small schools are different than others, and specifically 
whether small schools that were newly created, with the extra supports and rigorous application 
processes (and perhaps other differences), are more effective than the earlier generation of small 
schools. We examine the descriptive statistics on differences between the old small and new 
small schools in Table 4. The new small schools have smaller enrollment and more advantaged 
students than the old small schools. The students at the new small schools have similar levels of 
poverty (measured by free lunch eligibility) as the students at the old small schools, but the new 
small school students have a higher fraction of Asian and higher 8 th grade English and 
mathematics scores. 

To examine the potentially differential effectiveness of these different types of schools, 
we re-estimate the OLS and IV models separately for new and old small high schools. In Table 
3, the OLS estimates in Column (3) indicate that both new and old small schools have 
statistically significant positive effects on graduation, relative to large schools, but that the new 
small schools have a higher positive effect (0.125 vs. 0.072). The F-test statistics indicate that 
we reject (p < .01) the joint null hypotheses that the effect of the small schools is the same and 
that both have zero effect. 

Because the same endogenous school choice issue could exist for our two different 
school types as with the single small school type, we next turn to IV estimation. In column (4) 
of Table 3, we compute 2SLS estimates and instrument for both endogenous old small and new 
small high school attendance. The two first stages are reported in Appendix Table B.2. Since 
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this new specification includes multiple endogenous school choice variables, identification of the 
IV estimator requires multiple instruments. Our instruments are based on a set of distance based 
instrumental variables: student residence distance to old small schools, distance to new small 
schools, distance to old large schools, and distance new large schools. The first stage estimates 
indicate a negative and statistically significant relationship between distance from student 
residence to old small schools and old small school attendance and between distance to new 
small schools and new small school attendance. We explore below various other instrumental 
variable configurations and find our main estimate results are robust to alternative IV strategies. 

The results in Table 3 suggest that the OLS result is driven by selection on unobservable 
student characteristics as the IV estimates are considerably different from the corresponding OLS 
estimates. The IV results show that attending an old small high school had a negative effect on 
performance (p < .01), while attending a new small school had a positive effect (p < .01). The 
negative estimate for old small schools indicates that older generation of small schools were 
substantially worse than the large high schools (the omitted category of schools). Note that 
while this negative effect is quite sizable (-0.556 relative to large schools), the robustness 
analysis below suggests that the negative effect may be driven by a few outlier schools since we 
estimate a smaller negative effect with different cutoff values defining “small” schools (and thus 
large schools). The positive estimate for new small schools indicates a 17.5 percent gain in the 
probability of graduating from high school due to attending a new small school. The estimates 
thus reveal a ranking in the causal effectiveness of schools in producing rates of earned diplomas 
in four years: old small schools are the worst, large high schools are better than old small 
schools, and the new small schools are better than large high schools. 

Robustness 

Robustness: Classification of “Small” 

Table 5 checks the robustness of the main results from Table 3, Column 4, to various 
changes in variable definitions and sample selection. The first issue we address is the definition 
of “small.” Our current definition, classifying schools with enrollments of 550 or fewer as small, 
is within the range of the literature and policy initiatives but nonetheless somewhat arbitrary and, 
as the literature review indicated, larger size schools have been found to be effective as well. In 
order to test the robustness of the results of the basic model, we re-estimate the main model with 
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a cutoff of 600 (and then 650) students for a small school. As shown in Table 5, the positive new 
small school effect remains substantively unchanged, ranging between 14 and 18 percent, 
depending on the definition of small. Thus our main estimates of the positive effect of new small 
schools on graduation rates is largely robust to changes in the definition of “small” within a 
reasonable range. 

A more substantial change is that the negative effect of old small schools (relative to the 
omitted large school schools) is reduced considerably to about -0.388 to -0.339, depending 
on definition of small. As we change the definition of small, many formerly classified small 
schools are now classified as large schools and the relative effectiveness of school types reflects 
a different mixture of schools. The smaller negative effect of old small schools (relative to large) 
from changing the definition of small schools suggests there may be some particularly poorly 
performing old small schools with enrollments between 550 to 650 students, and these schools 
make a large contribution to the overall point estimates. 

Robustness: Excluding Selective High Schools 

Many of the high schools in NYC are selective, in that they require tests or auditions 
to attend. Well-known examples are Stuyvesant, Bronx Science, and Brooklyn Tech, but all 
together there are seven such high schools in the years of our sample. Column (3) of Table 5 
excludes these seven schools from the estimation sample and re-estimates the main model. The 
new small and old small effects are largely unchanged. 

Robustness: Constant Definition of Small 

In our main estimates, for both the 2007 and 2008 graduating cohorts, we classify each 
school as “small” by its current year enrollment (for the 9 th grade entering class). Some schools 
with enrollments near the cutoff of 550 switch classification from small to large in 2008. In 
Column (4) of Table 5 we test the robustness of our estimates to maintaining a constant 
definition of small by classifying schools as small using the 2007 enrollment only. 15 Our results 
remain nearly unchanged. 


15 There are twelve schools that are small in 2007 but not small in 2008. 
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Robustness: Alternative Instruments 

In Table 6, we test the robustness of our main results using a number of alternative IV 
strategies. Recall that our main specification includes distance to four types of schools: new and 
old large schools and new and old small schools, and the square of each of these distances. The 
estimates reported in Column (1) of Table 6 are from the 2SLS estimator excluding the four 
quadratic distance instruments and including only the linear distance instruments. In Column (2), 
we exclude the quadratic distance to large schools only. In Column (3), we exclude the quadratic 
instruments measuring distance to small schools. In Column (4), we expand the instrument set to 
include sixteen cohort specific instruments (each instrument is interacted with cohort, for 
example, distance to new small x 2007, distance to new small squared x 2007, etc). 

In Column (5) we do not use the distance to new and old large schools as excluded 
instruments but instead use these variables as control variables, which we include in the main 
outcome equation. Distance to small schools, conditional on distance to large school, is then 
capturing a student’s relative distance between small and large schools. This type of 
specification would provide a valid IV strategy in the case where one believed that only relative 
distance to school types was exogenous, but not absolute distance (see above for a discussion of 
IV validity and various tests we performed to check this validity). 

Overall our main estimates are robust to these alternative IV models. Across all of 
these alternative IV models, the causal effect of new small attendance ranges from 0.155 to 
0.199, with all of the estimates precisely estimated at least at the 10 percent level. The negative 
effect of old small school attendance shows more sensitivity to the IV estimator choice, with the 
negative effect relative to large schools ranging from -0.252 to -0.517, with the -0.252 effect 
estimated imprecisely. Across all models, however, the F-test statistic on the joint hypotheses 
that small and old schools have the same effect on graduation and the F-test statistic that old and 
new small schools jointly have a zero effect are generally large and we can reject these 
hypotheses at the 1 percent level. 

Other Outcomes 

In Table 7, we explore a number of different high school outcomes using our main IV 
specification, instrumenting for endogenous old small and new small school attendance. The 
omitted category is large high schools, so all effect estimates are gains or losses relative to 
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attending a large high school. In Column (1) we examine how attendance at old and new small 
schools affects the probability of earning a GED degree instead of a regular or Regents diploma 
(our “graduate” outcome) or not earning any degree (high school drop-out). The estimates reveal 
that attending a new small school lowers the probability of earning a GED by nearly 6 percent. 
This can be considered a desirable outcome combined with the positive 17 percent effect of new 
small school attendance on the probability of graduating. The estimates suggest that new small 
schools cause some students who would have otherwise earned a GED to instead graduate from 
high school. In contrast, the effect of old small school attendance has a small and imprecise 
effect on earning a GED. 

Earning a diploma in four years is a critical outcome, but the “quality” of the high school 
degree is important as well. Moreover, most states, New York included, are making passage of 
content examinations a requirement for graduation. To ascertain whether attending a small 
school has an effect on either the number of examinations taken or the scores achieved, we 
estimate IV results for two critical New York State Regents’ examinations - English and first 
level mathematics. 

Table 7 provides the IV estimates for a range of these examinations. New small high 
schools appear to be considerably more effective than the old small schools on all of these 
measures. Mirroring the effects on graduation, new small school attendance also has a 
statistically significant effect on attempting Regents mathematics and English examinations. Our 
IV estimates indicate that attending a new small school relative to a large school increases the 
probability of taking the Regents English by 14.4 percent and the Regents mathematics by 16.4 
percent. 

Perhaps because of the large effect on test taking, attendance at a new small schools is 
estimated to cause a reduction in average English scores and the likelihood of obtaining passing 
scores (for those who take the exam). New small schools attendance is estimated to reduce the 
student’s English score by 0.4 standard deviations and reduce the probability of obtaining a 
higher than 55 score by 4.5 percent and a higher than 65 score by 19.8 percent. While these 
results are seemingly in contradiction with the positive graduation effects, they may reflect the 
much higher proportion of students taking the English exam and that the marginal test taker had 
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lower ability. 16 Also recall that for our sample years, the minimum local diploma requires that 
only one Regents Examination be passed with a score of 55%. 

In addition, Table 7 reveals that the new small schools still have much better performance 
than old small schools in terms of test taking and performance, although we cannot reject the 
hypothesis at the 10 percent level that the effect of new and old small on average English scores 
is the same. 

In contrast to the English score results, we estimate that new small school attendance has 
a small negative and imprecise effect on math performance. Thus, while there is also a large and 
positive effect on test taking in mathematics as with English, there is no significant negative 
effect of new small school attendance on math performance. For old small schools, however, we 
estimate that attendance would not only reduce the probability of taking the mathematics exam 
but also substantially reduce the average score and probability of obtaining a passing score. 
Attendance at old small schools is estimated to reduce math scores by 0.517 standard deviations 
and the probability of earning greater than a 55 score is reduced by 8.6 percent and greater than a 
65 score by 43 percent. 

Student Sub-Group Analysis 

The literature on small high schools hypothesizes that particular benefits accrue to at- 
risk subgroups such as black, Hispanic, poor and/or male students. In Table 8, we present 
empirical evidence on graduation rates for subgroups attending NYC high schools. Based on IV 
regression results, the new small schools perfonn better than the old small schools for all 
subgroups except white students and better than large high schools for both girls and boys, 

Asian, Hispanic and non-poor students. For black and poor students, attending new small and 
large high schools is equivalent in terms of graduating in four years. Thus, students attending 
old small high schools clearly perfonn worse than those attending other size schools, while 
students attending new small high schools either perform better or equivalent to those attending 
large high schools. 


1,1 Because test taking is voluntary and likely related to unobservable variables that also affect high school choice, 
considering the test score outcomes introduces a “double endogneity” problem with both test taking and school 
choice being endogenous. While we have a plausible IV strategy for identifying school effects overall, we do not 
have a credible instrument for test taking. Understanding the important relationships between test taking and school 
attendance choices is an important area for future research. 
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DISCUSSION: WHY DOES “NEWNESS” MATTER? 


If only the new small high schools are more effective than large high schools, it is important 
to explore how the characteristics across types of schools might differ. We already noted the 
kinds of process differences that NYCDOE identified for new small schools, but are there also 
differences in the characteristics of the student bodies, in spending per pupil, or in teacher 
characteristics? Table 9 displays OLS results for a series of school-level variables and allows us 
to begin to answer this question. 

In terms of student racial composition and in comparison to old small schools, both the 
new and old small schools enroll lower percentages of Asian students. They do not differ from 
each other or large schools in terms of the percentages of enrolled black or Hispanic students, 
however. New small schools enroll fewer special education students compared to both large 
schools and old small schools and fewer limited English proficient (LEP) students compared to 
large schools (but not compared to old small schools). The percentage of poor students at new 
small schools is higher than at old small and large schools. 

Strikingly, resources for small high schools differ, with both new and old small 
schools receiving more funding for direct (school-level) resources and having lower class sizes 
(pupil-teacher ratios) compared to large schools. These differences are substantial (between 
$1500 and almost $2900 per pupil and around 2.7 fewer students per teacher). Small schools 
also have more inexperienced teachers and fewer with MA degrees (with new small significantly 
different from old small and large), which implies that the extra funding is financing lower class 
sizes and perhaps other services as well, but with less experienced and educated teachers. 

Finally, schools do not differ in terms of principals trained at the newly created Leadership 
Academy set up to provide good principals to school. 

Thus, the new small schools receive significantly more resources, work with less 
trained teachers, serve more poor but fewer special education and LEP students than large 
schools and sometimes than old small schools as well. These differences along with the 
processes the NYCDOE uses to choose among applicants proposing to form a new small school 
may set the stage for their success. 
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CONCLUSIONS 


District school reforms generally take several years to implement, and small high 
school reforms, which have been initiated in many major urban districts over the past two 
decades, are no different. In this paper, we use data on two cohorts of NYC high school students 
to estimate the effect of attending small high schools. A particularly important feature of this 
evaluation is that many small high schools existed from previous waves of creation in the 1990s, 
allowing us to test for the effect of smallness versus newness in school size. In order to ascertain 
if the size of the school is a cause of any changes in outcomes rather than just a product of the 
particular students who chose to attend these schools, we use instrumental variable estimators 
with plausibly exogenous distance from residence to school to correct for self-selection into 
schools. 

We find first that selection is likely to be a complicating factor as we find very 
different OLS and IV effects of small school attendance on student performance. Decomposing 
small high schools into their earlier wave (fully created before 2002) and a later wave, we find 
that the later, new small schools are the ones that are effective in terms of graduation as well as 
taking math and English examinations. Thus the findings on size itself are not clear cut. 
Attending small high schools can benefit students, but attending some small schools does not do 
so, on average. Only new small ones are effective (compared to large ones) and they are 
different from others. 

While the new and old small schools differ in a variety of ways, including greater 
resources per pupil, the most important differences may lie in NYCDOE institutional policies 
that govern their creation and practices such as the application process they must go through, the 
monetary and networking support received from non-profit organizations, the loosening of union 
rules in hiring teachers, and the temporary suspension of requirements to serve all special needs 
students. These differences raise a host of questions for policymakers in NYC and for those 
hoping to replicate the success in other districts. 

Perhaps most obvious and important are these: are these supports sustainable for the 
new small schools or can the schools be “weaned”? Can the successes of the new small schools 
be replicated in “old” small schools? Or is it the enthusiasm of teachers, leaders and staff 
energized by participating in building something “new” (or the ability of a new leader to choose 
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staff to meet a vision) that creates the effect, meaning “growing old” is the problem? Does the 
success hinge on the increased funding? If size itself is not the defining characteristic, then just 
reducing school size will not produce the same results. While the notion that “small size is not 
enough” seems (with the benefit of hindsight) to have significant intuitive appeal, urban districts 
around the country have jumped on the small school refonn bandwagon, replacing large, 
comprehensive schools with a myriad of small schools and schools-within-schools (with a wide 
range of resources, supports and characteristics) in the hopes of bolstering student performance. 
The evidence from New York City suggests that the success of these efforts will depend 
significantly on how those new schools are created and supported. 
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Table 1: Descriptive Statistics of NYC HS Students by School Size Category, 2008 
and 2009 Cohorts 



all 

small 

large 

% enrolled small schools 

20.26 

100 

— 

# of schools 

291 

169 

122 

distance to nearest small HS 

1.29 

0.73 

1.43 

distance to nearest large HS 

0.64 

0.59 

0.66 

Demographic Characteristics 




% Female 

53.36 

56.94 

52.45 

% Black 

37.25 

42.90 

35.82 

% Hispanic 

36.46 

42.28 

34.98 

% Asian 

14.46 

7.63 

16.20 

% White 

11.74 

7.04 

12.94 

% English is home language 

52.97 

59.22 

51.38 

% Overage 

16.61 

17.83 

16.31 

% Poor 

76.15 

77.85 

75.71 

% LEP 

3.25 

3.02 

3.31 

8th grade Math z-score 

0.000 

-0.129 

0.033 

8th grade ELA z-score 

0.000 

-0.101 

0.026 

Outcomes 




% Graduated 

65.76 

72.04 

64.16 

% Continued Enrollment 

21.74 

18.62 

22.53 

% Dropout 

9.84 

7.56 

10.43 

% GED 

2.66 

1.78 

2.88 

% Took Math Regents 

83.38 

87.88 

82.24 

% Score >55 

96.97 

96.51 

97.10 

% Score >65 

83.80 

80.14 

84.79 

% Took English Regents 

83.22 

86.99 

82.27 

% Score >55 

96.01 

95.49 

96.16 

% Score >65 

86.49 

83.39 

87.32 

Observations 

76,213 

15,444 

60,769 


Notes: Small schools are those with enrollments of 550 or fewer students. Distance is calculated 
using Euclidian distances. Poverty is defined by eligibility for free lunch. ELA and math z-scores are 
taken from the statewide ELA and math exams given in 8th grade and are standardized to have a 
mean of zero and a standard deviation of one. Regents exams are New York State exams given to 
high school students. Schools that are considered small in one year, but large in the other are 
reported twice in the "number of schools" row. 
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Table 2: Probability of Attending a Small School and Distance from Residence, OLS Regression 
Results 


Dependent variable: small school attendance 

( 1 ) 

( 2 ) 

Distance to nearest small school 

-0.044*** 

-0.115*** 


( 0 . 001 ) 

(0.004) 

Distance to nearest small school squared 


0.015*** 



( 0 . 001 ) 

Distance to nearest large school 

-0.009** 

- 0.012 


(0.004) 

( 0 . 010 ) 

Distance to nearest large school squared 


0.008 



(0.006) 

Year 2008 

0.028*** 

0.028*** 


(0.003) 

(0.003) 

Constant 

0.375*** 

0 41 1 *** 


(0.006) 

(0.007) 

Observations 

76,213 

76,213 

F - First Stage Excluded (2, 76193) 

817 47 *** 

— 

F - First Stage Excluded (4, 76193) 

— 

429.98*** 

F - Total Regression (19, 76193) 

527.13*** 

— 

F - Total Regression (21, 76193) 

— 

485.53*** 

R-squared 

0.094 

0.098 


Robust standard errors in parentheses (*** p<0.01, ** p<0.05, * p<0.1). All models control for gender, 
race\ethnicity, English proficiency, home language, overage for grade, poverty, performance on 8 th grade 
standardized EL A and math exams, and residence borough. Poverty is measured by eligibility for free lunch. Test 
scores are measured as z-scores with a mean of zero and a standard deviation of one. 
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Table 3: Probability of Graduating in Four Years, OLS and IV Regression Results 


Dependent variable: graduated 

(1) 

OLS 

(2) 

IV 

( 3 ) 

OLS 

( 4 ) 

IV 

Small 

0.108*** 

-0.011 




(0.011) 

(0.058) 



Old small 



0.072*** 

-0.556*** 




(0.016) 

(0.167) 

New small 



0.125*** 

0.175** 




(0.012) 

(0.084) 

Year 2008 

0.039*** 

0.043*** 

0.037*** 

0.026*** 


(0.005) 

(0.006) 

(0.005) 

(0.008) 

Constant 

0.664*** 

0.704*** 

0.663*** 

0.710*** 


(0.012) 

(0.023) 

(0.012) 

(0.029) 

Observations 

76,213 

76,213 

76,213 

76,213 

R-squared 

0.236 

0.227 

0.237 

0.131 

F (old sm = new sm) 



10.52*** 

13.84*** 

F (old sm = new sm = 0) 



58.42*** 

6.92*** 


Robust standard errors, adjusted for within-school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0.1). All 
models control for gender, race\ethnicity, English proficiency, home language, overage for grade, poverty, 
performance on 8 th grade standardized ELA and math exams, and residence borough. Poverty is measured by 


eligibility for free lunch. Test scores are measured as z-scores with a mean of zero and a standard deviation of one. 
Graduated is defined as earning a local. Regents, or Regents Elonors diploma, as defined by the New York State 
Department of Education. 
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Table 4: Descriptive statistics of New York City High School 


Students by New Small and Old Sma 

1 High School 


New Small 

Old Small 

% Enrolled 

13.83 

6.43 

# of Schools 

121 

48 

average size 

245 

420 

Demographic Characteristics 



% Female 

56.32 

58.27 

% Black 

43.20 

42.24 

% Hispanic 

40.62 

45.83 

% Asian 

8.67 

5.38 

% White 

7.33 

6.42 

% Overage 

17.25 

19.07 

% LEP 

2.98 

3.10 

% English is home language 

59.70 

58.19 

% Poor 

77.48 

78.65 

8th grade Math z-score 

-0.084 

-0.224 

8th grade ELA z-score 

-0.061 

-0.187 

Outcomes 



% Graduated 

74.83 

66.04 

% Continued Enrollment 

16.65 

22.86 

% Dropout 

6.96 

8.83 

% GED 

1.56 

2.26 


Notes: Small schools are those with enrollments of 550 or fewer students. 
Poverty is defined by eligibility for free lunch. ELA and math z-scores are 
taken from the statewide ELA and math exams given in 8th grade and are 
standardized to have a mean of zero and a standard deviation of one. New 
schools are schools with a graduating class in 2003; old schools are schools 
with graduating classes prior to 2003. 
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Table 5: Robustness Checks, Probability of Graduating, Old and New Small Schools, 
IV Regression Results 


Dependent 

variable: 

graduated 

( 1 ) 

Small <=600 

(2) 

Small <=650 

( 3 ) 

Excludes 
selective HS 

( 4 ) 

Constant 07 
def n small 

Old small 

-0.388*** 

-0.339*** 

-0.532*** 

-0.549*** 


(0.131) 

(0.118) 

(0.160) 

(0.166) 

New small 

0.142* 

0.185** 

0.173** 

0.190** 


(0.072) 

(0.079) 

(0.082) 

(0.095) 

Year 2008 

0.030*** 

0.029*** 

0.028*** 

0.027*** 


(0.007) 

(0.008) 

(0.009) 

(0.009) 

Constant 

0.705*** 

0.694*** 

0.708*** 

0.709*** 


(0.026) 

(0.027) 

(0.028) 

(0.029) 

Observations 

76,213 

76,213 

71,105 

76,213 

R-squared 

0.173 

0.175 

0.119 

0.124 

F (old=new) 

12 39*** 

13.50*** 

13.30*** 

12.62*** 

F (old=new=0) 

6.23*** 

g 1 q*** 

6.65*** 

6.36*** 


Robust standard errors, adjusted for within-school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0.1). All 
models control for gender, race\ethnicity, English proficiency, home language, overage for grade, poverty, 
performance on 8 th grade standardized ELA and math exams, and residence borough. Poverty is measured by 
eligibility for free lunch. Test scores are measured as z-scores with a mean of zero and a standard deviation of one. 
Graduated is defined as earning a local. Regents, or Regents Honors diploma, as defined by the New York State 
Department of Education. Selective high schools include schools such as Stuyvestant, Bronx Science, and Brooklyn 
Tech. Column (4) classifies the schools that are small in 2007, but have enrollments greater than 550 in 2008 as 
small in both years. 
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Table 6: Robustness checks, Probability of Graduating, Old and New Small Schools, IV 
Regression Results 


Dependent 

variable: 

graduated 

(1) 

No square 
instruments 

(2) 

No large sq 
instruments 

( 3 ) 

No small sq 
instruments 

( 4 ) 

Cohort specific 
instruments 

( 5 ) 

Distance to 
large on RHS 

Old small 

-0.252 

-0.517*** 

-0.310* 

-0.514*** 

-0.513*** 


(0.181) 

(0.162) 

(0.184) 

(0.157) 

(0.165) 

New small 

0.165** 

0.155* 

0.199** 

0.164** 

0.170* 


(0.083) 

(0.084) 

(0.082) 

(0.074) 

(0.092) 

Year 2008 

0.031*** 

0.028*** 

0.028*** 

0.027*** 

0.029*** 


(0.008) 

(0.008) 

(0.008) 

(0.007) 

(0.010) 

Constant 

0.684*** 

q | *** 

0.681*** 

0.708*** 

0.706*** 


(0.030) 

(0.028) 

(0.030) 

(0.027) 

(0.033) 

Observations 

76,213 

76,213 

76,213 

76,213 

76,213 

R-squared 

0.208 

0.145 

0.194 

0.146 

0.146 

F (old=new) 

4.57** 

1 2 90 *** 

6 90* * * 

J ^ J ^ ^ 

1 1.06*** 

F (old=new=0) 

3.09** 

5.96*** 

4 82*** 

1 

5.56*** 


Robust standard errors, adjusted for within-school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0.1). All 
models control for gender, race\ethnicity, English proficiency, home language, overage for grade, poverty, 
performance on 8 th grade standardized ELA and math exams, and residence borough. Poverty is measured by 
eligibility for free lunch. Test scores are measured as z-scores with a mean of zero and a standard deviation of one. 
Graduated is defined as earning a local. Regents, or Regents Honors diploma, as defined by the New York State 
Department of Education. The model in column (1) excludes the four quadratic instalments. Column (2) excludes 
the quadratic instruments measuring distance to large schools. Column (3) excludes the quadratic instruments 
measuring distance to small schools. Column (4) includes sixteen cohort specific instalments (each instalment is 
interacted with cohort, so distance to new small * 2007, distance to new small squared * 2007, etc). Column (5) 
includes the four measures capturing distance to the nearest large school (distance to nearest: new large, new large 
squared, old large, and old large squared) as independent variables. 
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Table 7: Other Education Outcomes, IV Regression Results 



( 1 ) 

GED 

( 2 ) 

Took 

English 

( 3 ) 

English 

>=55 

( 4 ) 

English 

>=65 

( 5 ) 

English 

z-score 

( 6 ) 

Took 

Math 

(7) 

Math 

>=55 

( 8 ) 

Math 

>=65 

( 9 ) 

Math 

z-score 

Old small 

-0.013 

-0.296*** 

-0.152*** 

-0.430*** 

-0.807** 

-0.251** 

-0.086** 

-0.431*** 

-0.517*** 


(0.032) 

(0.109) 

(0.051) 

(0.126) 

(0.346) 

(0.114) 

(0.037) 

(0.125) 

(0.196) 

New small 

-0.058*** 

0.144** 

-0.045** 

-0.198*** 

-0.402** 

0.164*** 

-0.016 

-0.066 

- 0.021 


( 0 . 021 ) 

(0.056) 

( 0 . 020 ) 

(0.059) 

(0.193) 

(0.061) 

(0.015) 

(0.054) 

( 0 . 100 ) 

Year 2008 

0.002 

0.034*** 

-0.003 

0 . 022 *** 

-0.131*** 

0.040*** 

-0.008*** 

0.009 

0.062*** 


( 0 . 002 ) 

(0.006) 

( 0 . 002 ) 

(0.007) 

(0.041) 

(0.007) 

( 0 . 002 ) 

(0.006) 

(0.013) 

Constant 

0.048*** 

0.847*** 

0.835*** 

1.006*** 

0.981*** 

0.622*** 

1.005*** 

0.934*** 

0.228*** 


(0.007) 

( 0 . 021 ) 

( 0 . 022 ) 

(0.009) 

(0.026) 

( 0 . 100 ) 

(0.006) 

(0.023) 

(0.038) 

Observations 

76,213 

76,213 

76,213 

63,427 

63,427 

63,427 

63,547 

63,547 

63,547 

R- squared 

0.001 

0.080 

0.089 

0.068 

0.089 

0.256 

0.084 

0.162 

0.480 

F (old=new) 

1.32 

12 45*** 

3.51* 

2.65 

1.36 

9 

2.68 

7.50*** 

5.12** 

F (old=new=0) 

3.84** 

6.72*** 

7 qg*** 

12.15*** 

3.84** 

5.65*** 

3.67** 

^ 44 ^ ^ ^ 

3.48** 


Robust standard errors, adjusted for within-school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0.1). All models control for gender, race\ethnicity, 
English proficiency, home language, overage for grade, poverty, performance on 8 th grade standardized ELA and math exams, and residence borough. Poverty is 
measured by eligibility for free lunch. Test scores are measured as z-scores with a mean of zero and a standard deviation of one. Graduated is defined as earning 
a local. Regents, or Regents Honors diploma, as defined by the New York State Department of Education. English and Math are two of the many NYS Regents 
examinations given three times annually. As students can re-take these exams, we use their most recent score. Columns (6) and (10) use student scores 
standardized to have a mean of zero and a standard deviation of one for all student taking the exam on that particular date. 
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Table 8: Subgroup Analyses, IV Regression Results 


Dependent 
variable: grad. 

(1) 

Female 

(2) 

Male 

(3) 

Asian 

(4) 

Black 

(5) 

Hispanic 

(6) 

White 

(V) 

Poor 

(8) 

Non-poor 

Old small 

-0.647*** 

-0.435** 

-0.281 

-0.563** 

-0.337** 

-0.092 

-0.462*** 

-0.437** 


(0.182) 

(0.192) 

(0.284) 

(0.243) 

(0.170) 

(0.251) 

(0.156) 

(0.218) 

New small 

0.177* 

0.162* 

0.241*** 

0.031 

0.316*** 

0.225* 

0.125 

0.228** 


(0.091) 

(0.098) 

(0.088) 

(0.112) 

(0.122) 

(0.125) 

(0.102) 

(0.093) 

Year 2008 

0.023** 

0.031*** 

0.000 

0.037*** 

0.031** 

0.033*** 

0.030*** 

0.026*** 


(0.009) 

(0.009) 

(0.007) 

(0.012) 

(0.012) 

(0.009) 

(0.010) 

(0.009) 

Constant 

0.798*** 

0.697*** 

0.756*** 

0.874*** 

0.633*** 

0.725*** 

0.643*** 

0.697*** 


(0.031) 

(0.033) 

(0.034) 

(0.040) 

(0.037) 

(0.037) 

(0.027) 

(0.037) 

Observations 

40,669 

35,544 

11,024 

28,390 

27,849 

8,950 

58,034 

18,179 

R-squared 

0.060 

0.180 

0.168 

0.107 

0.115 

0.221 

0.143 

0.114 

F (old=new) 

15.02*** 

6.61** 

3.69* 

4.92** 

y qy*** 

1.05 

8.22*** 

g 1 1 *** 

F (old=new=0) 

7 52 *** 

3.32** 

5.53*** 

2.71* 

4.34** 

1.63 

4.53** 

5.20*** 


Robust standard errors, adjusted for within-school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0.1). All models control for gender, race\ethnicity, 
English proficiency, home language, overage for grade, poverty, performance on 8 lh grade standardized ELA and math exams, and residence borough. Poverty is 
measured by eligibility for free lunch. Test scores are measured as z-scores with a mean of zero and a standard deviation of one. Graduated is defined as earning 
a local. Regents, or Regents Honors diploma, as defined by the New York State Department of Education. 
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Table 9: School Characteristics, OLS regression results 



( 1 ) 

% Asian 

( 2 ) 

% black 

( 3 ) 

% Hispanic 

( 4 ) 

% spec ed 

( 5 ) 

% LEP 

( 6 ) 

% poor 

(7) 

$ - direct 

( 8 ) 

Pupil-teacher 

Old small 

-7 777 *** 

3.464 

8.078** 

1.036 

-4.561*** 

6.446 

1,719.78*** 

-2.892*** 


(1.425) 

(4.060) 

(3.747) 

(0.998) 

(1.061) 

(4.051) 

(312.083) 

(0.418) 

New small 

-5.096*** 

6.331* 

4.877* 

-3.602*** 

-2.551** 

10.613*** 

2,894.71*** 

-2.815*** 


(1.691) 

(3.228) 

(2.860) 

(0.719) 

(1.232) 

(2.758) 

(273.788) 

(0.340) 

Constant 

11.673*** 

39.092*** 

37.002*** 

10.842*** 

10.754*** 

57.377*** 

9,568.04*** 

17 . 544 *** 


(1.228) 

(2.485) 

(2.166) 

(0.490) 

(0.825) 

(2.141) 

(169.018) 

(0.227) 

Observations 

278 

278 

278 

278 

278 

278 

276 

276 

R-squared 

0.061 

0.014 

0.020 

0.112 

0.036 

0.050 

0.298 

0.232 

F (old=new) 

3.84* 

0.56 

0.80 

20.81*** 

3.15* 

1.17 

1 1.98*** 

0.03 

F (old=new=0) 

15.02*** 

1.93 

2.68* 

16.69*** 

9.26*** 

7 4q=i=** 

58.09*** 

43.05*** 


Robust standard errors, adjusted for within-school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0.1). One school is missing all information in 2004 and 
2005. Two additional schools are missing spending and pupil-teacher information in 2004 and 2005. Twenty four schools are missing information about teachers. 
Column (9) reports the results for the percent of teachers who have been teaching at that school for less than two years. The percent of teachers with a masters 
degree or higher is shown in column (10). Teacher experience, in column (1 1) is defined as the percent of teachers at the school who have 5+ years of teaching 
experience. Column (12) reports the results for whether the school was ever run by a Leadership Academy principal. 
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Table 9 cont: School Characteristics, OLS regression results, 



(9) 

% tch < 2 yrs 

(10) 

% MA+ 

(11) 

tchr experience 

(12) 

LA principal 

Old small 

7.611*** 

-3.795** 

-11.338*** 

0.141** 


(2.692) 

(1.800) 

(2.235) 

(0.069) 

New small 

55.070*** 

-19.768*** 

-33.171*** 

0.040 


(1.934) 

(3.336) 

(2.426) 

(0.040) 

Constant 

40.310*** 

76.422*** 

55.604*** 

0.086*** 


(1.322) 

(0.662) 

(1.024) 

(0.026) 

Observations 

255 

255 

255 

279 

R-squared 

0.765 

0.164 

0.467 

0.021 

F (old=new) 

300.69*** 

18.91*** 

54 27*** 

2.06 

F (old=new=0) 

431.12*** 

18.97*** 

96.51*** 

2.23 


Robust standard errors, adjusted for within-school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0.1). One 
school is missing all information in 2004 and 2005. Two additional schools are missing spending and pupil-teacher 
information in 2004 and 2005. Twenty four schools are missing information about teachers. Column (9) reports the 
results for the percent of teachers who have been teaching at that school for less than two years. The percent of 
teachers with a masters degree or higher is shown in column (10). Teacher experience, in column (1 1) is defined as 
the percent of teachers at the school who have 5+ years of teaching experience. Column (12) reports the results for 
whether the school was ever run by a Leadership Academy principal. 
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Frequency Frequency 


Figure 1: Distribution of NYC high schools by enrollment 


Distribution of NYC high schools by enrollment, cohorts 2001 and 2002 
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Frequency 


Figure 2: Distribution of NYC high schools by graduation rates 


Distribution of graduation rates, cohorts 2007 and 2008 
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Appendix A: Regents Examinations 

The Regents Examinations are a series of tests, aligned with New York State’s Learning 
Standards, which New York students must pass in order to receive high school diplomas. They are 
designed and administered under the authority of the Board of Regents of the University of The State Of 
New York (the State governing body for K-16 education) and prepared by teacher examination 
committees and testing specialists. Examination scores range from 0%-100%. 

To earn a Regents high school diploma, New York students need to obtain appropriate credits in a 
number of specific subjects by passing year-long or half-year courses, after which they must pass a 
Regents examination in that subject area. This expectation is in addition to passing the courses 
themselves, the passing grade of which is based on an individual teacher's or school's own tests and class 
work. Starting with the cohort entering grade 9 in 200 1 , and thus including our own cohorts, to receive a 
Regents high school diploma students need to score a 65 or above in the following five content areas: 
Integrated Algebra (or Math A), Global History and Geography, U.S. History and Government, 
Comprehensive English, and any one science area. 17 To earn an Advanced Regents diploma, students take 
additional credits in a foreign language, pass an additional Regents exam in science (at least one course in 
life science and one in physical science), and pass a second Regents exam in math. Students in our 
cohorts also were allowed to graduate with local (not Regents) diplomas, which required passing any one 
of five Regents examinations with a score of at least 55%. 18 

The math exams offered for the cohorts in our study are Math A and Math B. Topics tested by the 

Math A Regents exam include equations and inequalities, probability and statistics and geometry. Math 

B, which is optional, is taken after the student has passed Math A. Topics that can be tested include 

concepts from trigonometry and advanced algebra, as well as some pre-calculus and calculus. 19 

New York State’s science core curricula include Living Environment, Physical Setting/Earth Science, 
Physical Setting/Chemistry, and Physical Setting/Physics. All students entering grade 9 in 2001 must earn 
three units of credit in science although they must pass only one Regents examination in science to obtain 
a Regents diploma. The three science credits must be comprised of commencement-level science courses, 
including one course from the Physical Setting (physical science) and one course from the Living Environment (life 
science). The third may be from either life sciences or physical sciences. All commencement-level science courses, 
including specialized courses, must include laboratory activities. 20 


17 http://schools.nyc.gOv/NR/rdonlyres/53FADF0D-D784-435E-8675-90E20624DAEl/0/201 lGenEd.pdf 

18 The local diploma option was gradually made more difficult (more examinations and higher grades required) and 
was finally removed as an option for students entering 9 th grade in 2008 and later. 

19 The New York State Math A and Math B Regents Examinations are no longer a part of the High School 
Mathematics curriculum. The last administration of the Regents Examination in Mathematics A was January 2009 
and the last administration of the Regents Examination in Mathematics B was June 2010. These exams were 
replaced by three exams: Integrated Algebra I, Geometry and Algebra II, and Trigonometry. 

20 For more information, see http://www.p 1 2,nysed. gov/part 1 00/pages/ 1 005 .html#regentsdiploma 
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Appendix B: Tables -- Appendix Table B.l: Definition of Variables 


Variable 

Definition 

Distance Variables 


Distance to the nearest small high 
school 

The distance between the nearest small high school address and 
student’s residence zip code in 8 lh grade 21 

Distance to the nearest large high 
school 

The distance between the nearest large high school address and 
student’s residence zip code in 8 th grade 22 

Student Demographics 


female 

female=l, male=0 

black 

black=l, non-black=0 

Asian 

Asian=l, non-Asian=0 

white 

white=l, non-white=0 

Hispanic 

Hispanic=l, non-Hispanic=0 

overage 

Overage for student’s expected graduation date 

LEP 

Limited English Proficiency status, based on 9 th grade year 

English Home Language 

English is the home language, based on 9 th grade year 

poor 

Eligible for free lunch in 8 th grade 23 

ELA 

Standardized (z) 8 th grade English Language Arts test score 

math 

Standardized (z) 8 th grade math test score 

Manhattan 

8 th grade residence: Manhattan=l, other borough=0 

Bronx 

8 th grade residence: Bronx=l, other borough=0 

Brooklyn 

8 th grade residence: Brooklyn=l, other borough=0 

Queens 

8 th grade residence: Queens=l, other borough=0 

Student Outcomes 


graduated 

Student earned a local. Regents, or Regents Honors diploma within 
four years 

continued enrollment 

Student is still enrolled after four years of high school 

dropout 

Student dropped out of high school 

GED 

Student earned a GED 

took math 

Student took the math sequential 1 Regents Exam 

took English 

Student took the English Regents Exam 

math >=55 

Student scored a 55 or greater on the math sequential 1 Regents 
Exam 

math >=65 

Student scored a 65 or greater on the math sequential 1 Regents 
Exam 

English >= 55 

Student scored a 55 or greater on the English Regents Exam 

English >= 65 

Student scored a 65 or greater on the English Regents Exam 

Cohort ID 


year 200X or cohort 200X 

Student is expected to graduate high school in year (cohort) 200X 
and is in 8 th grade five years earlier 

School Characteristics 


small school 

550 students or fewer enrolled, based on enrollment during the 
student’s 9 th grade year 

Small*200X 

Small school in student’s 9 lh grade year, for students expected to 
graduate in 200X 

large 

The school with greater than 550 students enrolled, based on 
enrollment during the student’s 9 th grade year 

new 

School that had a graduating class starting in 2003 

old 

School that had a graduating class before 2003 


12 For Cohort 2008, distances are created using student’s 7 th grade residence zip code due to data unavailability in 
8 th grade. 

23 The 7 th grade free lunch variable is used when 8 th grade variable is unavailable. 
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Leadership Academy Principal 

Indicates that a school had a principal who graduated from the 
Leadership Academy for at least one year between 2005 and 2008 

Percent black 

Percent of black or African American students during 9 th grade 
year 

Percent Hispanic 

Percent of Hispanic and Latino students during 9 th grade year 

Percent Asian 

Percent Asian or Native Hawaiian/Other Pacific Islander students 
during 9 lh grade year 

Percent Special Education 

Percent of students receiving special education services during 9 th 
grade year 

Percent Limited English proficient 

Percent of limited English proficient students during 9 th grade year 

Percent Poor 

Percent of students eligible for free lunch during 9 th grade year 

Spending on direct services 

Dollars spent on services provided directly to public school students 
and staff, and which take place primarily in the school building 
during the school day during the school year during 9 th grade year 

Student to Teacher Ratio 

Pupil to teacher ratio in 9 th grade 

Percent Teaching <= 2 years 

Percent of teachers teaching in that school for less than two years 
during 9 th grade year 

Percent Teacher Masters+ 

Percentage of teachers in the school with Master’s Degrees or 
higher during 9 th grade year 

Percent Teacher Experience 

The percent of teachers in the school who have been teaching for 
five plus years during 9 th grade year. 
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Appendix Table B.2: First stage, likelihood of attending a small high school 



(1) 

Small 

(2) 

Old small 

(3) 

New small 

Distance to nearest old small school 

-0.055*** 

-0.050*** 

-0.005* 


(0.003) 

(0.002) 

(0.003) 

Distance to nearest old small school squared 

0.005*** 

0.006*** 

-0.001 


(0.001) 

(0.000) 

(0.000) 

Distance to nearest new small school 

-0.071*** 

0.010*** 

-0.081*** 


(0.004) 

(0.003) 

(0.004) 

Distance to nearest new small school squared 

0.008*** 

-0.002*** 

0.010*** 


(0.001) 

(0.000) 

(0.001) 

Distance to nearest old large school 

0.005 

-0.003 

0.008 


(0.010) 

(0.006) 

(0.009) 

Distance to nearest old large school squared 

-0.003 

0.001 

-0.005 


(0.005) 

(0.003) 

(0.004) 

Distance to nearest new large school 

0.003 

0.009*** 

-0.006*** 


(0.002) 

(0.001) 

(0.002) 

Distance to nearest new large school squared 

0.001*** 

-0.001*** 

0.001*** 


(0.000) 

(0.000) 

(0.000) 

Constant 

0.426*** 

0.120*** 

0.307*** 


(0.008) 

(0.005) 

(0.007) 

Observations 

76,213 

76,213 

76,213 

F - First Stage Excluded (8, 76187) 

243.30*** 

85.64*** 

206.16*** 

F - Total Regression (25, 76187) 

426.28*** 

135.86*** 

260.00*** 

R-squared 

0.102 

0.049 

0.073 


This is the first stage for Table X in text. Column (1) is the first stage for Column (2) of Table X, and Columns (2) 
and (3) are the first stages for Column (4) of Table X. Robust standard errors, adjusted for within-school clusters, in 
parentheses (*** p<0.01, ** p<0.05, * p<0.1). All models control for gender, race\ethnicity, English proficiency, 
home language, overage for grade, poverty, performance on 8 th grade standardized ELA and math exams, and 
residence borough. Poverty is measured by eligibility for free lunch. Test scores are measured as z-scores with a 
mean of zero and a standard deviation of one. 
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Appendix Table B.3: Relationship between minimum distance to small schools and average student characteristics, by residence zip 
code 



(1) 

Distance to nearest 
small school 

(2) 

Distance to nearest 
new small school 

(3) 

Distance to nearest 
old small school 

Female 

0.870 

-0.298 

1.025 


(0.793) 

(0.502) 

(0.653) 

Asian 

-0.031 

0.097 

-0.070 


(0.872) 

(0.603) 

(0.723) 

Black 

-0.205 

-0.858 

1.442** 


(0.852) 

(0.623) 

(0.593) 

White 

1.413 

-0.241 

1.236* 


(0.949) 

(0.670) 

(0.634) 

Overage 

-0.706 

-0.724 

0.310 


(1.141) 

(0.820) 

(1.347) 

LEP 

2.108 

1.900 

-3.441 


(3.785) 

(2.728) 

(4.072) 

English is home language 

-0.168 

0.232 

-0.653 


(0.758) 

(0.565) 

(0.622) 

Poor 

-0.184 

-0.470 

0.392 

Average 8 th grade ELA z-score 

(0.755) 

(0.490) 

(0.786) 

-0.585 

0.195 

-0.850 

Average 8 th grade math z-score 

(0.465) 

(0.362) 

(0.539) 

0.619 

-0.460 

1.307*** 


(0.462) 

(0.356) 

(0.477) 

Distance to nearest large school 

0.641*** 

0.200* 

0.250 


(0.155) 

(0.113) 

(0.217) 

Distance to nearest old small school 

— 

0.478*** 

— 


— 

(0.050) 

— 

Distance to nearest new small school 

--- 

— 

1.082*** 


— 

— 

(0.086) 

Manhattan 

-0.473*** 

0.216 

-0.845*** 


(0.170) 

(0.158) 

(0.239) 

Brooklyn 

0.850*** 

0.458** 

-0.141 


(0.247) 

(0.186) 

(0.298) 

Queens 

0.882*** 

0.936*** 

-0.852*** 


(0.217) 

(0.162) 

(0.281) 

Constant 

0.069 

0.695 

-0.447 


(0.916) 

(0.657) 

(1.156) 

Observations 

170 

170 

170 

R-squared 

0.496 

0.778 

0.737 


Robust standard errors in parentheses (*** p<0.01, ** p<0.05, * p<0.1) 
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Appendix Table B. 4: Full OLS and IV regression results 


Dependent variable: graduated 

(1) 

OLS 

(2) 

IV 

( 3 ) 

OLS 

( 4 ) 

IV 

Small School 

0.108*** 

-0.011 




(0.011) 

(0.058) 



Old small 



0.072*** 

-0.556*** 




(0.016) 

(0.167) 

New small 



0.125*** 

0.175** 




(0.012) 

(0.085) 

Year 2008 

0.039*** 

0.043*** 

0.037*** 

0.026*** 


(0.005) 

(0.006) 

(0.005) 

(0.008) 

Female 

0.069*** 

0.071*** 

0.069*** 

0.075*** 


(0.005) 

(0.005) 

(0.005) 

(0.005) 

Black 

0.068*** 

0.069*** 

0.068*** 

0.070*** 


(0.007) 

(0.008) 

(0.007) 

(0.010) 

Asian 

0.085*** 

0.079*** 

0.084*** 

0.066*** 


(0.007) 

(0.008) 

(0.007) 

(0.010) 

White 

0.061*** 

0.054*** 

0.061*** 

0.049*** 


(0.009) 

(0.009) 

(0.009) 

(0.012) 

Overage 

-0.129*** 

-0.131*** 

-0.129*** 

-0.129*** 


(0.005) 

(0.005) 

(0.005) 

(0.005) 

LEP 

0.066*** 

0.063*** 

0.065*** 

0.051*** 


(0.012) 

(0.013) 

(0.012) 

(0.013) 

English Spoken at Home 

-0.053*** 

-0.048*** 

-0.053*** 

-0.047*** 


(0.006) 

(0.006) 

(0.006) 

(0.007) 

Poor 

-0.064*** 

-0.066*** 

-0.064*** 

-0.066*** 


(0.005) 

(0.006) 

(0.005) 

(0.006) 

ELA z-score 

0.075*** 

0.076*** 

0.075*** 

0.073*** 


(0.003) 

(0.003) 

(0.003) 

(0.004) 

ELA z-score squared 

-0.004** 

-0.004** 

-0.004** 

-0.003* 


(0.002) 

(0.002) 

(0.002) 

(0.002) 

Math z-score 

0.133*** 

0.132*** 

0.133*** 

0.129*** 


(0.005) 

(0.006) 

(0.005) 

(0.006) 

Math z-score squared 

0.006** 

0.006** 

0.006** 

0.006** 


(0.003) 

(0.003) 

(0.003) 

(0.003) 

ELA*Math 

-0.033*** 

-0.034*** 

-0.033*** 

-0.035*** 


(0.003) 

(0.003) 

(0.003) 

(0.003) 

Manhattan 

-0.017 

-0.020 

-0.013 

0.036 


(0.016) 

(0.019) 

(0.016) 

(0.034) 

Queens 

0.000 

-0.025 

0.002 

-0.006 


(0.012) 

(0.018) 

(0.012) 

(0.027) 

Brooklyn 

-0.028** 

-0.055*** 

-0.027** 

-0.057** 


(0.012) 

(0.020) 

(0.012) 

(0.026) 

Constant 

0.664*** 

0.704*** 

0.663*** 

0.710*** 


(0.012) 

(0.023) 

(0.012) 

(0.029) 

Observations 

76,213 

76,213 

76,213 

76,213 

R-squared 

0.236 

0.227 

0.237 

0.131 


Robust standard errors, adjusted for within school clusters, in parentheses (*** p<0.01, ** p<0.05, * p<0. 1) 
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